2026-04-10 18:30:07,037 [ 277314 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 18:30:07,038 [ 277314 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 18:30:07,038 [ 277314 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 18:30:07,038 [ 277314 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_8irp2d --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_polymorphic_parts/test.py::test_compact_parts_only test_format_avro_confluent/test.py::test_select test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_executable_dictionary/test.py::test_executable_input_bash test_log_query_probability/test.py::test_log_quries_probability_one test_kerberos_auth/test.py::test_bad_path_to_keytab test_max_authentication_methods_per_user/test.py::test_alter test_password_constraints/test.py::test_complexity_rules test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] [gw6] [ 1%] PASSED test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting [gw7] [ 2%] PASSED test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type [gw7] [ 3%] PASSED test_password_constraints/test.py::test_default_password_type [gw9] [ 4%] PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab [gw9] [ 5%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw6] [ 6%] PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create [gw9] [ 7%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw6] [ 8%] PASSED test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw6] [ 9%] PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw5] [ 10%] PASSED test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw5] [ 11%] PASSED test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python [gw4] [ 12%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw5] [ 13%] PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python [gw5] [ 14%] PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load [gw4] [ 15%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw4] [ 16%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw5] [ 17%] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python [gw5] [ 18%] PASSED test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash [gw2] [ 19%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] [gw5] [ 20%] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] [gw1] [ 21%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] [gw4] [ 22%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw5] [ 23%] PASSED test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader [gw3] [ 24%] PASSED test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth [gw5] [ 25%] PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python [gw3] [ 26%] PASSED test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded [gw3] [ 27%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw3] [ 28%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw4] [ 29%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] [gw2] [ 30%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] [gw5] [ 31%] PASSED test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python [gw4] [ 32%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw7] [ 33%] PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw0] [ 34%] PASSED test_polymorphic_parts/test.py::test_compact_parts_only test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] [gw4] [ 35%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw2] [ 36%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] [gw1] [ 37%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] [gw0] [ 38%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] test_keeper_profiler/test.py::test_profiler [gw8] [ 39%] PASSED test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two [gw4] [ 40%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw0] [ 41%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] [gw5] [ 42%] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python [gw4] [ 43%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] [gw9] [ 44%] FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader [gw2] [ 45%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] test_profile_events_s3/test.py::test_profile_events test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] [gw7] [ 46%] PASSED test_keeper_profiler/test.py::test_profiler [gw4] [ 47%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw1] [ 48%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 49%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] [gw4] [ 50%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw4] [ 51%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw2] [ 52%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] [gw1] [ 53%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] [gw5] [ 54%] PASSED test_profile_events_s3/test.py::test_profile_events test_match_process_uid_against_data_owner/test.py::test_different_user [gw2] [ 55%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] [gw6] [ 56%] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_log_levels_update/test.py::test_log_levels_update [gw2] [ 57%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] [gw1] [ 58%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] [gw0] [ 59%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] [gw2] [ 60%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] [gw6] [ 61%] PASSED test_log_levels_update/test.py::test_log_levels_update [gw9] [ 62%] PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw7] [ 63%] PASSED test_match_process_uid_against_data_owner/test.py::test_different_user test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache [gw2] [ 64%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] [gw1] [ 65%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 66%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] [gw1] [ 67%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] [gw2] [ 68%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] [gw6] [ 69%] SKIPPED test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache test_parallel_replicas_increase_error_count/test.py::test_increase_error_count [gw2] [ 70%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] [gw1] [ 71%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] [gw0] [ 72%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] test_polymorphic_parts/test.py::test_polymorphic_parts_index [gw0] [ 73%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw2] [ 74%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] [gw0] [ 75%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw2] [ 76%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] [gw1] [ 77%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 78%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] [gw9] [ 79%] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count [gw1] [ 80%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] [gw2] [ 81%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] [gw2] [ 82%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] [gw1] [ 83%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] [gw2] [ 84%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] [gw2] [ 85%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] [gw1] [ 86%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] [gw2] [ 87%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] [gw2] [ 88%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] [gw2] [ 89%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] [gw2] [ 90%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] [gw2] [ 91%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] [gw2] [ 92%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] [gw2] [ 93%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] [gw2] [ 94%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] [gw2] [ 95%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] [gw2] [ 96%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] [gw2] [ 97%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw2] [ 98%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw8] [ 99%] PASSED test_log_query_probability/test.py::test_log_quries_probability_two test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant [gw8] [100%] PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant =================================== FAILURES =================================== ______________________ test_reconfig_remove_2_and_leader _______________________ [gw9] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_reconfig_remove_2_and_leader(started_cluster): """ Remove 2 followers from a cluster of 5. Remove leader from 3 nodes. """ global zk1, zk2, zk3, zk4, zk5 zk1 = create_client(node1) config = ku.get_config_str(zk1) print("Initial config", config) assert len(config.split("\n")) == 5 for i in range(100): zk1.create(f"/test_two_{i}", "somedata") zk4 = create_client(node4) zk4.sync("/test_two_0") ku.wait_configs_equal(config, zk4) zk5 = create_client(node5) zk5.sync("/test_two_0") ku.wait_configs_equal(config, zk5) for i in range(100): assert zk4.exists(f"test_two_{i}") assert zk5.exists(f"test_two_{i}") zk4.create(f"/test_two_{100 + i}", "otherdata") zk2 = create_client(node2) config = zk2.reconfig(joining=None, leaving="4,5", new_members=None) print("After removing 4,5", config) assert len(config.split("\n")) == 3 assert "node1" in config assert "node2" in config assert "node3" in config assert "node4" not in config assert "node5" not in config zk1.stop() zk1 = create_client(node1) zk1.sync("/test_two_0") ku.wait_configs_equal(config, zk1) for i in range(200): assert zk1.exists(f"test_two_{i}") assert zk2.exists(f"test_two_{i}") assert not node1.contains_in_log(log_msg_removed) assert not node2.contains_in_log(log_msg_removed) assert not node3.contains_in_log(log_msg_removed) assert node4.contains_in_log(log_msg_removed) assert node5.contains_in_log(log_msg_removed) > assert ku.is_leader(cluster, node1) E assert False E + where False = (, ) E + where = ku.is_leader test_keeper_reconfig_remove_many/test.py:100: AssertionError ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-04-10 18:31:05.974000 [ 706 ] INFO : Running tests in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/test.py (cluster.py:2748, start) 2026-04-10 18:31:05.975000 [ 706 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 18:31:06.067000 [ 706 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw9 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:31:06.124000 [ 706 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:31:06.206000 [ 706 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw9 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:31:06.206000 [ 706 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:31:06.258000 [ 706 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw9 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:31:06.304000 [ 706 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:31:06.371000 [ 706 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw9 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:31:06.372000 [ 706 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw9-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:31:06.433000 [ 706 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:31:06.434000 [ 706 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw9 (cluster.py:889, cleanup) 2026-04-10 18:31:06.434000 [ 706 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:31:06.472000 [ 706 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:31:06.472000 [ 706 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:31:06.604000 [ 706 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:31:06.604000 [ 706 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:31:06.604000 [ 706 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:31:06.605000 [ 706 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:31:06.639000 [ 706 ] DEBUG : Stdout:6 (cluster.py:145, run_and_check) 2026-04-10 18:31:06.640000 [ 706 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:31:06.687000 [ 706 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:31:06.687000 [ 706 ] DEBUG : Volumes pruned: 6 (cluster.py:925, cleanup) 2026-04-10 18:31:06.687000 [ 706 ] DEBUG : Setup directory for instance: node1 (cluster.py:2768, start) 2026-04-10 18:31:06.690000 [ 706 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:31:06.690000 [ 706 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:31:06.691000 [ 706 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:31:06.693000 [ 706 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:31:06.695000 [ 706 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper1.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:31:06.697000 [ 706 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/database (cluster.py:4772, create_dir) 2026-04-10 18:31:06.697000 [ 706 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/logs (cluster.py:4783, create_dir) 2026-04-10 18:31:06.698000 [ 706 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:31:06.698000 [ 706 ] DEBUG : Setup directory for instance: node2 (cluster.py:2768, start) 2026-04-10 18:31:06.699000 [ 706 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:31:06.699000 [ 706 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:31:06.700000 [ 706 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:31:06.701000 [ 706 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:31:06.701000 [ 706 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper2.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:31:06.702000 [ 706 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/database (cluster.py:4772, create_dir) 2026-04-10 18:31:06.702000 [ 706 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/logs (cluster.py:4783, create_dir) 2026-04-10 18:31:06.702000 [ 706 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:31:06.703000 [ 706 ] DEBUG : Setup directory for instance: node3 (cluster.py:2768, start) 2026-04-10 18:31:06.703000 [ 706 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:31:06.704000 [ 706 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:31:06.704000 [ 706 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:31:06.705000 [ 706 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:31:06.706000 [ 706 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper3.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:31:06.706000 [ 706 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/database (cluster.py:4772, create_dir) 2026-04-10 18:31:06.707000 [ 706 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/logs (cluster.py:4783, create_dir) 2026-04-10 18:31:06.707000 [ 706 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:31:06.707000 [ 706 ] DEBUG : Setup directory for instance: node4 (cluster.py:2768, start) 2026-04-10 18:31:06.708000 [ 706 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:31:06.708000 [ 706 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:31:06.709000 [ 706 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:31:06.709000 [ 706 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:31:06.710000 [ 706 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper4.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:31:06.710000 [ 706 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/database (cluster.py:4772, create_dir) 2026-04-10 18:31:06.711000 [ 706 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/logs (cluster.py:4783, create_dir) 2026-04-10 18:31:06.711000 [ 706 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:31:06.711000 [ 706 ] DEBUG : Setup directory for instance: node5 (cluster.py:2768, start) 2026-04-10 18:31:06.712000 [ 706 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:31:06.712000 [ 706 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:31:06.713000 [ 706 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:31:06.713000 [ 706 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:31:06.714000 [ 706 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper5.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:31:06.715000 [ 706 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/database (cluster.py:4772, create_dir) 2026-04-10 18:31:06.715000 [ 706 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/logs (cluster.py:4783, create_dir) 2026-04-10 18:31:06.715000 [ 706 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:31:06.716000 [ 706 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env (cluster.py:96, _create_env_file) 2026-04-10 18:31:06.722000 [ 706 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:31:06.723000 [ 706 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:31:06.723000 [ 706 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:31:06.724000 [ 706 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:31:06.754000 [ 706 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 18:31:06.757000 [ 706 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env --project-name roottestkeeperreconfigremovemany-gw9 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 18:31:17.377000 [ 706 ] DEBUG : Stderr: node2 Skipped - Image is already being pulled by node1 (cluster.py:147, run_and_check) 2026-04-10 18:31:17.378000 [ 706 ] DEBUG : Stderr: node3 Skipped - Image is already being pulled by node1 (cluster.py:147, run_and_check) 2026-04-10 18:31:17.378000 [ 706 ] DEBUG : Stderr: node4 Skipped - Image is already being pulled by node1 (cluster.py:147, run_and_check) 2026-04-10 18:31:17.378000 [ 706 ] DEBUG : Stderr: node5 Skipped - Image is already being pulled by node1 (cluster.py:147, run_and_check) 2026-04-10 18:31:17.378000 [ 706 ] DEBUG : Stderr: node1 Pulling (cluster.py:147, run_and_check) 2026-04-10 18:31:17.378000 [ 706 ] DEBUG : Stderr: node1 Pulled (cluster.py:147, run_and_check) 2026-04-10 18:31:17.379000 [ 706 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env --project-name roottestkeeperreconfigremovemany-gw9 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 18:31:17.379000 [ 706 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env --project-name roottestkeeperreconfigremovemany-gw9 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:31:19.238000 [ 706 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw9_default Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.238000 [ 706 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw9_default Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.238000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.239000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:31:19.240000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:31:19.241000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:31:19.241000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:31:19.241000 [ 706 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:31:19.241000 [ 706 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:19.274000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.297000 [ 706 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:31:19.304000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.306000 [ 706 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.9.5... (cluster.py:3165, start) 2026-04-10 18:31:19.313000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.322000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.427000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.531000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.639000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.744000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.849000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:19.953000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.060000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.165000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.271000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.375000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.487000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.591000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.694000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.798000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:20.903000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.010000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.114000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.218000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.321000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.428000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.535000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.640000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.746000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.851000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:21.959000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.066000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.171000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.285000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.413000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.523000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.629000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.738000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.845000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:22.949000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.052000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.169000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.293000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.405000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.521000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.628000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.748000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.855000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:23.958000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.063000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.174000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.283000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.387000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.491000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.596000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.707000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.847000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:24.956000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:25.061000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.193000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.301000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.419000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.530000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.639000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.747000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.856000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.970000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/e64cc93700c841a151d304889d778053359691a80324152e408bfe54872407c7/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:27.972000 [ 706 ] DEBUG : ClickHouse node1 started (cluster.py:3169, start) 2026-04-10 18:31:27.972000 [ 706 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:28.005000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.006000 [ 706 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:31:28.009000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.009000 [ 706 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.9.4... (cluster.py:3165, start) 2026-04-10 18:31:28.011000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.019000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.123000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.228000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.336000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.477000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.586000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.691000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.799000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:28.913000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.021000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.131000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.239000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/791787ce058957afc1d08e7664f156d91dc29ec68970775c475466f7c9f85151/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.244000 [ 706 ] DEBUG : ClickHouse node2 started (cluster.py:3169, start) 2026-04-10 18:31:29.244000 [ 706 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:29.253000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.254000 [ 706 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:31:29.264000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.266000 [ 706 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.9.3... (cluster.py:3165, start) 2026-04-10 18:31:29.272000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.276000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/94492986c415310dcf5e766186d8019f7119139caf0525965bfafd51e35a8fed/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.277000 [ 706 ] DEBUG : ClickHouse node3 started (cluster.py:3169, start) 2026-04-10 18:31:29.277000 [ 706 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:29.284000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.285000 [ 706 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:31:29.352000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.353000 [ 706 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.9.2... (cluster.py:3165, start) 2026-04-10 18:31:29.358000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.375000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/a479bcae97e87f2ef077ab26be12020294d68447b20d4f7055fb25a33dc4fce4/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.377000 [ 706 ] DEBUG : ClickHouse node4 started (cluster.py:3169, start) 2026-04-10 18:31:29.378000 [ 706 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:29.418000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.419000 [ 706 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:31:29.423000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.429000 [ 706 ] DEBUG : Waiting for ClickHouse start in node5, ip: 172.16.9.6... (cluster.py:3165, start) 2026-04-10 18:31:29.436000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.440000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/25dfcac37a848f9fe87fb8efce6e757e7fd3e659bd54eb63eb79d5d6549a2f30/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:29.441000 [ 706 ] DEBUG : ClickHouse node5 started (cluster.py:3169, start) ----------------------------- Captured stdout call ----------------------------- Initial config server.1=node1:9234;participant;1 server.2=node2:9234;participant;1 server.3=node3:9234;participant;1 server.4=node4:9234;participant;1 server.5=node5:9234;participant;1 After removing 4,5 server.1=node1:9234;participant;1 server.2=node2:9234;participant;1 server.3=node3:9234;participant;1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:31:29.515000 [ 706 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:29.521000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:32.997000 [ 706 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:33.001000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:33.293000 [ 706 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:33.305000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:43.074000 [ 706 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:43.079000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:43.614000 [ 706 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:43.619000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:31:45.535000 [ 706 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw9-node1-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2061, exec_in_container) 2026-04-10 18:31:45.536000 [ 706 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw9-node1-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:31:46.178000 [ 706 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw9-node2-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2061, exec_in_container) 2026-04-10 18:31:46.178000 [ 706 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw9-node2-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:31:48.859000 [ 706 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw9-node3-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2061, exec_in_container) 2026-04-10 18:31:48.860000 [ 706 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw9-node3-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:31:49.277000 [ 706 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw9-node4-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2061, exec_in_container) 2026-04-10 18:31:49.278000 [ 706 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw9-node4-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:31:49.653000 [ 706 ] DEBUG : Stdout:/var/log/clickhouse-server/clickhouse-server.log:2026.04.10 18:31:43.564563 [ 601 ] {} RaftInstance: this server (4) has been removed from the cluster, will step down itself soon. config log idx 271 (cluster.py:145, run_and_check) 2026-04-10 18:31:49.653000 [ 706 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw9-node5-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2061, exec_in_container) 2026-04-10 18:31:49.654000 [ 706 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw9-node5-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:31:50.135000 [ 706 ] DEBUG : Stdout:/var/log/clickhouse-server/clickhouse-server.log:2026.04.10 18:31:43.641140 [ 595 ] {} RaftInstance: this server (5) has been removed from the cluster, will step down itself soon. config log idx 274 (cluster.py:145, run_and_check) 2026-04-10 18:31:50.136000 [ 706 ] DEBUG : Sending stat to :9181 (keeper_utils.py:271, send_4lw_cmd) 2026-04-10 18:31:50.136000 [ 706 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:31:50.145000 [ 706 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw9-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:31:52.185000 [ 706 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env --project-name roottestkeeperreconfigremovemany-gw9 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.633000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:20.637000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:20.638000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:20.638000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:20.638000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:20.638000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:20.638000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:20.639000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:20.639000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:20.639000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:20.640000 [ 706 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.686000 [ 706 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.709000 [ 706 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.726000 [ 706 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.743000 [ 706 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:32:20.759000 [ 706 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/.env --project-name roottestkeeperreconfigremovemany-gw9 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw9/node5/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:32:21.763000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:21.763000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.764000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:21.773000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.773000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:21.773000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.773000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node4-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node5-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.774000 [ 706 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw9-node1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.775000 [ 706 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw9_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:32:21.775000 [ 706 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw9_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:32:21.777000 [ 706 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:32:21.871000 [ 706 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw9 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:32:21.918000 [ 706 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:32:21.976000 [ 706 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw9 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:32:21.977000 [ 706 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw9-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:32:22.018000 [ 706 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:32:22.018000 [ 706 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw9 (cluster.py:889, cleanup) 2026-04-10 18:32:22.019000 [ 706 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:32:22.058000 [ 706 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:32:22.059000 [ 706 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:32:22.100000 [ 706 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:32:22.101000 [ 706 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:32:22.101000 [ 706 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:32:22.101000 [ 706 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:32:22.166000 [ 706 ] DEBUG : Stdout:5 (cluster.py:145, run_and_check) 2026-04-10 18:32:22.166000 [ 706 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:32:22.222000 [ 706 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:32:22.223000 [ 706 ] DEBUG : Volumes pruned: 5 (cluster.py:925, cleanup) ----------------- generated report log file: parallel1_0.jsonl ----------------- ============================== slowest durations =============================== 451.49s call test_log_query_probability/test.py::test_log_quries_probability_two 96.99s call test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 68.14s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 65.68s call test_log_query_probability/test.py::test_log_quries_probability_one 51.34s setup test_format_avro_confluent/test.py::test_select 42.81s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 38.58s call test_polymorphic_parts/test.py::test_compact_parts_only 38.55s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 36.48s setup test_keeper_profiler/test.py::test_profiler 35.81s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 34.73s setup test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 31.48s teardown test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 31.43s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 29.62s teardown test_format_avro_confluent/test.py::test_select_auth_encoded_complex 29.39s teardown test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 27.59s setup test_polymorphic_parts/test.py::test_compact_parts_only 26.42s call test_executable_dictionary/test.py::test_executable_source_updated_python 25.37s setup test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 24.25s setup test_executable_dictionary/test.py::test_executable_input_bash 23.97s setup test_profile_events_s3/test.py::test_profile_events 23.93s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 23.53s setup test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 23.30s teardown test_keeper_profiler/test.py::test_profiler 23.19s teardown test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 22.88s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 22.63s teardown test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 22.36s teardown test_profile_events_s3/test.py::test_profile_events 20.93s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 20.74s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 20.66s call test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 20.20s setup test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 19.15s setup test_kerberos_auth/test.py::test_bad_path_to_keytab 18.63s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 17.88s call test_profile_events_s3/test.py::test_profile_events 16.51s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 16.12s setup test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 15.81s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 15.68s setup test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 15.61s setup test_log_query_probability/test.py::test_log_quries_probability_one 15.21s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 15.05s setup test_max_authentication_methods_per_user/test.py::test_alter 14.87s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 14.82s setup test_password_constraints/test.py::test_complexity_rules 14.66s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 14.45s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 14.39s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 14.08s teardown test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 14.05s setup test_match_process_uid_against_data_owner/test.py::test_different_user 13.75s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 13.54s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 13.06s setup test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 12.80s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 11.96s call test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 11.87s call test_executable_dictionary/test.py::test_executable_input_slow_python 11.63s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 11.59s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 11.51s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 11.26s call test_match_process_uid_against_data_owner/test.py::test_different_user 11.11s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 10.98s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 10.63s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 9.61s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 9.44s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 9.11s teardown test_executable_dictionary/test.py::test_executable_source_updated_python 9.07s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 8.55s teardown test_log_query_probability/test.py::test_log_quries_probability_two 8.43s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 8.39s call test_executable_dictionary/test.py::test_executable_source_python 8.09s teardown test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 7.84s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 7.61s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 7.48s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 7.45s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 7.21s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 6.99s teardown test_max_authentication_methods_per_user/test.py::test_create_default_setting 6.93s setup test_log_levels_update/test.py::test_log_levels_update 6.51s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 6.21s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 6.07s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 6.06s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 5.89s teardown test_password_constraints/test.py::test_default_password_type 5.83s call test_log_levels_update/test.py::test_log_levels_update 5.79s call test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 5.69s call test_executable_dictionary/test.py::test_executable_source_argument_python 5.64s teardown test_log_levels_update/test.py::test_log_levels_update 5.56s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 5.51s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 5.21s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 5.12s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 5.00s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 4.94s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 4.79s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 4.73s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 4.72s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 4.71s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 4.58s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 4.55s call test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 4.42s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 4.40s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 4.23s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 4.22s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 4.20s call test_password_constraints/test.py::test_complexity_rules 4.19s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 4.10s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 3.98s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 3.96s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 3.92s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 3.91s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 3.91s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 3.88s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 3.85s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 3.75s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 3.75s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 3.75s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 3.71s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 3.70s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 3.70s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 3.66s call test_max_authentication_methods_per_user/test.py::test_alter 3.65s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 3.61s teardown test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 3.56s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 3.55s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 3.52s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 3.50s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 3.39s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 3.33s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 3.29s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 3.29s call test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 3.25s call test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 3.20s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 3.19s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 3.14s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 2.89s teardown test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 2.84s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 2.83s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 2.82s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 2.78s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 2.77s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 2.63s call test_max_authentication_methods_per_user/test.py::test_alter_default_setting 2.58s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 2.57s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 2.57s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 2.56s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 2.54s call test_executable_dictionary/test.py::test_executable_source_exit_code_check 2.52s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 2.47s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 2.47s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 2.42s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 2.42s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 2.32s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 2.32s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 2.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 2.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 2.07s call test_polymorphic_parts/test.py::test_polymorphic_parts_index 1.72s call test_executable_dictionary/test.py::test_executable_input_python 1.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 1.46s call test_max_authentication_methods_per_user/test.py::test_create_default_setting 1.44s call test_executable_dictionary/test.py::test_executable_input_sum_python 1.44s teardown test_match_process_uid_against_data_owner/test.py::test_different_user 1.42s call test_executable_dictionary/test.py::test_executable_non_direct_input_bash 1.28s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 1.26s call test_executable_dictionary/test.py::test_executable_input_signalled_python 1.23s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 1.17s call test_executable_dictionary/test.py::test_executable_input_bash 1.15s call test_kerberos_auth/test.py::test_bad_path_to_keytab 1.14s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 1.07s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 1.07s call test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 1.00s call test_max_authentication_methods_per_user/test.py::test_create 0.97s call test_format_avro_confluent/test.py::test_select_auth 0.96s call test_format_avro_confluent/test.py::test_select 0.90s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 0.90s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 0.85s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 0.84s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 0.84s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 0.83s call test_password_constraints/test.py::test_default_password_type 0.80s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 0.79s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 0.77s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 0.75s call test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.74s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 0.74s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 0.74s call test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.74s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 0.73s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 0.70s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 0.69s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 0.68s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 0.64s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 0.56s call test_keeper_profiler/test.py::test_profiler 0.54s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 0.52s call test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 0.22s call test_format_avro_confluent/test.py::test_select_auth_encoded 0.19s call test_format_avro_confluent/test.py::test_select_auth_encoded_complex 0.02s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.02s teardown test_polymorphic_parts/test.py::test_compact_parts_only 0.01s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.01s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.01s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 0.01s teardown test_log_query_probability/test.py::test_log_quries_probability_one 0.01s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.01s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_log_query_probability/test.py::test_log_quries_probability_two 0.00s teardown test_format_avro_confluent/test.py::test_select 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s setup test_executable_dictionary/test.py::test_executable_input_sum_python 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s teardown test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_format_avro_confluent/test.py::test_select_auth 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_executable_dictionary/test.py::test_executable_source_updated_python 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_executable_dictionary/test.py::test_executable_source_python 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.00s teardown test_executable_dictionary/test.py::test_executable_source_python 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_format_avro_confluent/test.py::test_select_auth_encoded 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.00s teardown test_kerberos_auth/test.py::test_bad_path_to_keytab 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_executable_dictionary/test.py::test_executable_source_argument_python 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s teardown test_format_avro_confluent/test.py::test_select_auth 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_executable_dictionary/test.py::test_executable_input_bash 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_format_avro_confluent/test.py::test_select_auth_encoded_complex 0.00s setup test_executable_dictionary/test.py::test_executable_input_python 0.00s teardown test_password_constraints/test.py::test_complexity_rules 0.00s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s teardown test_executable_dictionary/test.py::test_executable_source_exit_code_check 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter 0.00s setup test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s setup test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s teardown test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.00s setup test_password_constraints/test.py::test_default_password_type 0.00s setup test_executable_dictionary/test.py::test_executable_source_exit_code_check 0.00s teardown test_max_authentication_methods_per_user/test.py::test_create 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s teardown test_executable_dictionary/test.py::test_executable_input_sum_python 0.00s setup test_max_authentication_methods_per_user/test.py::test_create_default_setting 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s setup test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s setup test_max_authentication_methods_per_user/test.py::test_create 0.00s setup test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s teardown test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s teardown test_executable_dictionary/test.py::test_executable_source_argument_python 0.00s teardown test_format_avro_confluent/test.py::test_select_auth_encoded 0.00s teardown test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_python =========================== short test summary info ============================ FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader PASSED test_max_authentication_methods_per_user/test.py::test_alter PASSED test_password_constraints/test.py::test_complexity_rules PASSED test_password_constraints/test.py::test_default_password_type PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab PASSED test_max_authentication_methods_per_user/test.py::test_create PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting PASSED test_executable_dictionary/test.py::test_executable_input_bash PASSED test_executable_dictionary/test.py::test_executable_input_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python PASSED test_executable_dictionary/test.py::test_executable_input_sum_python PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_executable_dictionary/test.py::test_executable_source_argument_python PASSED test_format_avro_confluent/test.py::test_select PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check PASSED test_format_avro_confluent/test.py::test_select_auth PASSED test_format_avro_confluent/test.py::test_select_auth_encoded PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] PASSED test_executable_dictionary/test.py::test_executable_source_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table PASSED test_polymorphic_parts/test.py::test_compact_parts_only PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] PASSED test_log_query_probability/test.py::test_log_quries_probability_one PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] PASSED test_keeper_profiler/test.py::test_profiler PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_profile_events_s3/test.py::test_profile_events PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] PASSED test_log_levels_update/test.py::test_log_levels_update PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 PASSED test_match_process_uid_against_data_owner/test.py::test_different_user PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] PASSED test_log_query_probability/test.py::test_log_quries_probability_two PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant SKIPPED [1] test_merge_tree_check_part_with_cache/test.py:28: Skip with debug build and sanitizers. This test manually corrupts cache which triggers LOGICAL_ERROR and leads to crash with those builds ============= 1 failed, 98 passed, 1 skipped in 564.64s (0:09:24) ============== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_8irp2d --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.